An Iterated Conditional Modes/Medians Algorithm for Empirical Bayes Selection of Massive Variables

Empirical Bayes methods are privileged in data mining because they can absorb prior information on model parameters and are free of choosing tuning parameters. We proposed an iterated conditional modes/medians (ICM/M) algorithm to implement empirical Bayes selection of massive variables while incorporating sparsity or more complicated a priori information. The algorithm is constructed on the basis of iteratively minimizing a conditional loss function. The iterative conditional modes are employed to obtain data-driven estimates of hyperparameters, and the iterative conditional medians are used to estimate the model coefficients and therefore enable the selection of massive variables. The ICM/M algorithm is computationally fast, and can easily extend the empirical Bayes thresholding, which is adaptive to parameter sparsity, to complex data. Empirical studies suggest very competitive performance of the proposed method, even in the simple case of selecting massive regression predictors.